Видео с ютуба Sparse Transformers

Sparse Transformers and MuseNet | AISC

Giannis Daras: Improving sparse transformer models for efficient self-attention (spaCy IRL 2019)

Объяснение принципа разреженного внимания DeepSeek: на 80% дешевле ИИ с длинным контекстом

What are Sparse Transformers?

A Window Into LLMs | Sparse Autoencoders Explained

Attention in transformers, step-by-step | Deep Learning Chapter 6

Sparse LLMs at inference: 6x faster transformers! | DEJAVU paper explained

Sparse is Enough in Scaling Transformers (aka Terraformer) | ML Research Paper Explained

BigBird Research Ep. 3 - Block Sparse Attention, ITC vs. ETC

Weight-sparse transformers have interpretable circuits (OpenAI Research)

Is Sparse Attention more Interpretable?

BigBird Transformer за 3 минуты! | Разреженное внимание, теория графов и линейные трансформеры

Sparse Expert Models (Switch Transformers, GLAM, and more... w/ the Authors)

LLM - Dense vs Sparse Attention Explained

Soft Mixture of Experts - An Efficient Sparse Transformer

SAF3R - Dynamic Sparse Attention for Feed-Forward 3D Reconstruction Transformers

Как внимание стало настолько эффективным [GQA/MLA/DSA]

CVPR2023 Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers

Kaggle Reading Group: Generating Long Sequences with Sparse Transformers | Kaggle